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Abstract 

The purpose of this longitudinal study was to explore the impact of a direct reading 
instruction program, Reading Mastery ( RM ), on literacy performance of students over a five-year 
period. Initially, 700 students in grades K-3 were randomly selected from six elementary schools 
to participate in the study; an additional 100 randomly-selected kindergarten students were added 
to the sample for each subsequent year of the study. 

Literacy performance was assessed using oral and silent reading performance measures 
from the Qualitative Reading Inventory-3 ( QRI ), reading scale scores from the Mississippi 
Curriculum Test ( MCT ), and cloze scores from the Hunter & Grundin Literacy Profiles ( HGLP ). 

Because all students in the school system were receiving RM no comparison group was 
available to aid interpretation of the findings. In an attempt to overcome this difficulty, data were 
analyzed in three parts. First, only data for students who began RM in kindergarten were examined 
descriptively and graphically. Second, statistical comparisons by grade level were made between 
students who began RM in kindergarten and those who began RM in a later grade. Third, 
descriptive and graphical comparisons of group performance overtime were made between groups 
of students who began RM at different grade levels. 

Although a few statistically significant differences were found between students who 
began receiving RM instruction in kindergarten and those who began receiving RM at a later 
grade, the preponderance of data provided little evidence to support the viability of RM for 



improving reading comprehension. 
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Reading Performance of Elementary Students: Results of a 
Five-Year Longitudinal Study of Direct Reading Instruction 

In response to the “No Child Left Behind” federal legislation, some school districts have 
implemented Direct Instruction (DI) for teaching reading, even though inconsistent findings of the 
effects of DI on students’ reading comprehension have been reported in the literature (Abt 
Associates, 1977; Benbow, 1974; Bruton & Owen, 1988; Contreras, 1980; House, Glass, McLean, 
and Walker, 1978; Kennedy, 1978; Kuder, 1990; McCabe, 1974; McGlotten, 1982; Meyer, 1984; 
Mosley, 1997; O’Connor, Jenkins, Cole, & Mills, 1993; Slavin, Karweit, and Madden, 1989; 
Stallings, 1975; Stebbins, St. Pierre, Proper, Anderson, & Cerva, 1977). 

Results from the initial Project Follow Through study, the largest reported study of the 
longitudinal effects of DI on beginning reading instruction via a systematic-code approach (i.e., 
synthetic phonics) from kindergarten through third grade, indicated that the reading performance 
of students who received DI was at the 41 st percentile, nine percentile points below the median 
(Stebbins, St. Pierre, Proper, Anderson, & Cerva, 1977), even though the students taught by DI 
had higher beginning reading achievement scores on the Metropolitan Achievement Test ( MAT) 
than those in the control group (Abt Associates, 1977; Stallings, 1975). There were inconsistent 
findings reported, however, across school settings (House, Glass, McLean, and Walker, 1978; 
Kennedy, 1978). A re-analysis of the Project Follow Through data compared the average DI 
schools’ effect size on the MAT subscale scores with the control schools’ average effect size and 
reported that the average effect size difference across MAT basic skills was 1.8, with an average 
effect size for MAT total reading scores of DI schools being 1.6 and an average effect size for total 
reading scores of the control schools being .75 (Bereiter & Kurland, 1981-82). 

The findings from some studies have not favored DI in the areas of phonics (Benbow, 
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1974; O’Connor, Jenkins, Cole, & Mills, 1993), reading comprehension (Bruton & Owen, 1988; 
Contreras, 1980; Kuder, 1990; McGlotten, 1982; Mosley, 1997), basic concepts and vocabulary 
(McCabe, 1974; McGlotten, 1982), and reading achievement (McGlotten, 1982). 

Other studies of the effects of DI have reported mixed results. McCabe (1974) compared 
the effects of DISTAR and traditional early reading programs and reported that pre-kindergarten 
and kindergarten students in traditional programs did significantly better than DISTAR students on 
the Peabody Picture Vocabulary Test, but scores of first-grade students taught by DISTAR were 
higher than students in basal instruction; on the Wide Range Achievement Test the scores of first- 
grade students who received basal reading instruction were significantly higher than scores of 
DISTAR students, while scores of kindergartners taught by DISTAR were significantly higher than 
those of kindergartners taught by reading readiness materials. Scarcelli (1999) found that DRI 
instruction for first-grade students who were in the average-to-below-average range of reading 
ability produced significantly higher scores on the Gates-MacGinitie Reading Test than scores of 
students receiving whole language instruction, but no significant differences on the measure were 
found for students in the middle-third and upper-third ranges of reading ability. Carnine, Camine, 
and Gersten (1984) reported that first-grade students taught by DISTAR did not perform well when 
decoding un-taught words, but that third-grade students taught by DISTAR demonstrated 
integration of phonic and contextual strategies during oral reading. 

A meta-analysis of 15 studies (Adams & Engelmann, 1996), including only those studies 
that used complete DI programs developed by Engelmann and associates, indicated an effect size 
of .69 for reading (based on 43 comparisons across the 15 studies). This finding, however, should 
be considered in light of the study’s limitations: only four of the 15 studies specifically examined 
reading comprehension (the remainder examined vocabulary /language development and word 
recognition); only three studies contained procedures that attempted to ensure fidelity of 
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implementation of DI; only two studies included more than 60 students; 12 studies included only 
special education students; and five of the studies examined DI that was in place less than a year. 
In addition, Adams and Engelmann ignored the weak reading comprehension effect (.07) reported 
by Slavin, Karweit, and Madden (1989). 

Promoters of DI programs for teaching reading believe that “. . . virtually all the reading 
failure in the early grades could be avoided if teachers . . . were given well-constructed code- 
emphasis instructional materials” to teach reading during the beginning stages of reading (Carnine, 
Silbert, & Kameenui, 1997, p. 56). This belief by advocates of DI has encouraged the production 
of large numbers of commercial materials designed to develop phonological decoding processes, 
some of which are designed to supplement a school’s adopted reading program. Also, there are 
commercially-produced developmental reading programs that include a strong phonological 
decoding component, such as Reading Mastery , Rainbow Edition, developed by Engelmann and 
Hammer (1995). 

Reading Mastery (RM) is one of several DISTAR programs ( DISTAR is an acronym for 
Direct Instruction System for Teaching Arithmetic and Reading, which eventually became Direct 
Instruction System for Teaching and Remediation) developed by Engelmann and his colleagues 
(Engelmann and Hammer, 1995). RM lessons exemplify DI principles: (a) teacher-directed, 
scripted lessons; (b) presentation of phonological decoding skills (synthetic phonics) follows a 
hierarchical sequence; (c) student mastery is required at each step; (d) students respond orally as 
individuals, as a choral group, or by completing workbook assignments; and (e) systematic 
practice and review with a range of examples provide opportunities to correct student errors 
immediately. 

Most of the published studies of the effects of DI, using the RM program or some other 
commercial program, have included fewer than 100 students and only a few studies have included 
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samples of students who received more than a year of DI. Also, the inconsistent findings reported 
in the research literature related to the effects of DI, the lack of research examining the impact of 
DI on reading comprehension, and the lack of longitudinal studies of the effects of DI indicated 
the need for further study. This study presents the findings from a five-year longitudinal study 
that was designed to explore how sequential, systematic direct instruction impacts students’ 
reading comprehension as they progress through the elementary grades. The major research 
question addressed by the study was: How does sequential, systematic direct instruction in 
reading via the Reading Mastery ( RM ) program impact students’ reading comprehension over 
time, when considering grade level at which RM began and length of time that students received 
RM ? 

Method 

Participants and School Setting 

In August 2000 the target school district implemented direct instruction in reading in 
grades K-6 in all six elementary schools via the Reading Mastery ( RM) program (1995). Because 
of inconsistent findings in published studies of the effects of direct reading instruction, the 
superintendent felt that it was important to examine the effects of RM over a five-year period. 
Although the school district administration believed that the use of sequential, systematic direct 
instruction would positively influence the reading skills of students, the district requested that 
researchers from a nearby university conduct an independent, longitudinal study of the impact of 
RM on students’ reading comprehension scores. It was agreed that the measures of reading 
comprehension performance would be selected by the researchers. 

Students enrolled in the school district’s six elementary schools when the study began 
represented four racial groups: African American (approximately 89%), European American 
(approximately 9%), Hispanic (approximately 1%), and Pacific Rim (slightly less than 1%). 
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Approximately 2,900 students were enrolled in the six elementary schools at the initiation of the 
study period, with approximately 89% of the students being from low socioeconomic 
backgrounds. 

Prior to the implementation of the RM program, students in grades K-6 received traditional 
instruction, with kindergarten students being taught through learning centers designed to develop 
language/vocabulary skills, alphabet/phonics skills, social skills, age-appropriate mathematics, 
science, and social studies concepts, and interest in children’s books; in grades 1-6, commercial 
basal programs were used to teach reading, including the Harcourt Brace Jovanovich Reading 
Program, Laureate Edition (Cullinan el al., 1989) and the Macmillan/McGraw-Hill Reading 
Program, Spotlight on Literacy (Aoki el al., 1997). The chart presented below shows the 
instructional programs received by participants prior to implementation of RM (i.e., first-, second-, 
and third-grade participants the first year of the study). 

Upon initiation of this longitudinal study, 700 students enrolled in grades K, 1,2, and 3 
(approximately 175 students per grade level) were randomly selected from the six elementary 
schools to participate in the study, with an additional 100 randomly-selected kindergarten students 
being added to the sample each subsequent year of the study. Thus, the sample contained eight 
groups of participants who had received varied amounts of RM instruction (i.e., four groups had 
five years of RM\ one group had four years of RM: one group had three years of RM\ one group 
had two years of RM; one group had one year of RM). Because RM was taught in all six 
elementary schools, it was not possible to form a control group of students from the school 
district. 

Based on the grade at which each group of participants began receiving RM, the grade- 
level composition for each of the eight groups across the five-year period of the RM program is 



shown in Figure 1 . 
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Grade 


Year 1 


Year 2 


Year 3 


Year 4 


Year 5 


K 


K:/?M 


K: RM 


K: RM 


K: RM 


K: RM 


1 


1: /?M; K: traditional 


1 :RM; K: RM 


1: RM ; K: AM 


1: AM; K: RM 


1: RM; K: RM 


2 


2: /?M; 1: basal; 
K: traditional 


2: RM ; 1: KM; 

K: traditional 


2: AM; 1: KM; 
K:RM 


2: AM; 1: AM; 
K: RM 


2: RM; 1: AM; 
K: AM 


3 


3: RM; 2: basal; 
1: basal; 

K: traditional 


3: AM; 2: KM; 

1: basal; 

K: traditional 


3: KM; 2: AM; 

1: /?M; K: traditional 


3: KM; 2: RM; 
1: AM; K: AM 


3: RM; 2: KM; 
1: RM; K: AM 


4 




4: RM ; 3: KM; 

2: basal; 1: basal; 
K: traditional 


4: KM; 3: RM; 
2: /?M; 1: basal; 
K: traditional 


4: /fM; 3: /?M; 
2: RM; 1: KM; 

K: traditional 


4: KM; 3: RM; 
2: AM; 1: RM; 
K: RM 


5 






5: KM; 4:/?M; 

3: RM; 2: basal; 
1: basal; 

K: traditional 


5: KM; 4: AM; 
3: RM; 2: RM; 

1: basal; 

K: traditional 


5 : RM; 4: RM; 
3: RM; 2: RM; 
1 :RM; 

K: traditional 


6 








6: RM; 5: RM; 
4: RM; 3: RM; 

2:: basal; 1: basal; 
K: traditional 


6: RM; 5: RM; 
4: RM; 3: RM; 
2: RM; 1: basal; 
K: traditional 



7 7:RM; 6: RM ; 

5: RM ; 4: KM; 

3: RM; 2: basal; 
1: basal; 

K: traditional 
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At the end of the five-year of the study, complete sets of reading comprehension data 
were available for eight groups of participants, with group numbers ranging from 48 to 433, as 
follows: four groups of participants who began RM instruction in the first year of the study at 
different grade levels (i.e., K, 1,2, or 3) with each group having completed five years of RM: and 
four groups of participants who began receiving RM instruction in the second, third, fourth, and 
fifth years of the study with each group having completed four, three, two, and one years of RM , 
respectively (, Figure 1 ). 

Assessments of Reading Performance 

Two assessments were used to measure students’ reading comprehension performance: 
a) the Hunter-Grundin Literacy Profiles ( Hunter- Grundin & Grundin, 1990), and b) the 
Qualitative Reading Inventory-3 (Leslie & Caldwell, 2001). In addition, reading scale scores 
from the Mississippi Curriculum Test (2003), a state-adopted instrument, were used as the 
measures of students’ general reading performance. 

The Hunter-Grundin Literacy Profiles ( HGLP ) use a cloze procedure to measure reading 
comprehension. The HGLP consists of grade-level stories, which were drawn from classic 
literature. The first sentence of each story is left intact, with deletions in the remainder of the 
story occurring in a set pattern (e.g., after every 3-5 words of connected text); for each deleted 
word, there is a group of four words from which to select the one word that will give meaning to 
the text. A student’s raw score on a story is the number of accurate words that the student 
selected for deleted words in the story. Establishment of lower- and upper-limit raw score 
boundaries of a given passage is determined by multiplying the total number of deletions in a 
passage by .40 and by .60, respectively, thus indicating the score boundaries for a passage 
considered to be at the student’s instructional reading level; raw scores below the lower limit 
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indicate that the passage is at the student’s frustration reading level, and raw scores above the 
upper limit indicate that the passage is at the student’s independent reading level. 

The levels of the HGLP were standardized based on a sample of students enrolled in 
more than 70 schools, representing 23 different local education authorities, in England, Scotland, 
and Wales; the sample of students represented inner-city, suburban, small town, and rural areas, 
with the socioeconomic range of the students being from low to middle class including students 
from Educational Priority Area schools (the composition of these schools included student 
populations in which 50% or more were from immigrant families) and from schools whose 
student populations were composed almost exclusively of middle-class backgrounds. The 
authors reported that the HGLP was highly correlated (r = .87) with the Neale Analysis of 
Reading Ability measure, a standardized measure of reading comprehension (Hunter-Grundin & 
Grundin, 1979). 

The Qualitative Reading Inventory-3 ( QRI-3 ) was designed “. . . to provide diagnostic 
information about (1) conditions under which students can identify words and comprehend text 
successfully, and (2) conditions that appear to result in unsuccessful word identification, 
decoding, and/or comprehension” (Leslie & Caldwell, 2001, p. 1). The QRI-3 provides graded 
passages, from pre-primer through high school levels; for pre-primer through sixth-grade levels 
both narrative and expository passages are included, and for the upper-middle-school and high- 
school grades the passages are drawn from the content areas of literature, social studies, and 
science. The grade-level passages are used to analyze oral miscues and to assess reading 
comprehension. 

Using the graded passages, independent, instructional, and frustration reading levels may 
be determined for the student. The criteria associated with each of the three reading levels are as 
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follows: (1) independent level = word identification in context (oral reading of graded passages) 
is 98% or higher, and comprehension is 90% or higher; (2) instructional level = word 
identification in context (oral reading of graded passages) is 90% to 97% for total accuracy, and 
comprehension is 70% to 89%; and (3) frustration level = word identification in context (oral 
reading of graded passages) is less than 90% for total accuracy, and comprehension is less than 
70%. The formulae used to determine percentages are: (1) word identification in context = 
(number of words in passage minus number of miscues) divided by the number of words in the 
passage; and (2) comprehension = number of questions correctly answered divided by total 
number of questions with the result multiplied by 100. 

As a passage is being read aloud by the student, miscues are marked by the examiner on 
the examiner’s copy of the passage; a tape recording of the student’s reading is recommended so 
that the examiner can ascertain that all miscues are marked on the examiner’s copy. Any oral- 
reading deviation from the printed passage is considered a miscue (i.e., insertions, omissions, 
substitutions, reversals, and self-corrections); the total numbers of miscues per type of miscue 
are recorded by examiner. The QRI-3 includes a Miscue Analysis Worksheet which allows the 
examiner to qualitatively analyze the student’s miscues. After all miscues have been recorded on 
the worksheet, the examiner totals the columns for graphic similarity-initial, graphic similarity- 
final, semantically acceptable, and self-corrections; these totals permit the examiner to analyze 
the word identification strategies used by the student. By dividing the total number of miscues 
into each column total and multiplying the result by 100, the examiner can determine the percent 
of miscues represented by each column. For example, if a student made a total of 27 miscues, of 
which 8 contained graphically similar letters in the final position, the percent of miscues that 




Direct Reading Instruction, 12 



were graphically similar in the final position would be 29%. The manual of the QRI-3 provides 
guidelines for interpreting the results of the Miscue Analysis Worksheet. 

Comprehension assessment may be done orally and/or silently, by asking the student to 
retell the content of the passage and/or by asking questions about the content of the passage. The 
percent of questions answered correctly by the student is used to determine the student’s reading 
performance level on the passage (i.e., independent, instructional, or frustration level). Since the 
retelling procedure was not used in this study, the question procedures are described here. After 
the student has read the passage, the examiner asks the questions that accompany the passage; 
the questions are of two types, explicit and implicit. Explicit questions require answers that were 
stated directly in the passage, while implicit questions require that the student make use of clues 
that were in the passage in order to derive inferences from having read the passage. For 
example, an explicit question might be: Where did Mark and his brother go after the ballgame?, 
and an implicit question might be: How do we know that Mark cared about his brother? The 
manual of the QRI-3 recommends that “. . . answers be scored as either right or wrong with no 
half points given” (p. 73). Credit may be given, however, for “. . . any answer that includes the 
same information in different words” (p. 73). Prior knowledge may not be used to answer 
explicit questions, as the answer must have been included in the passage. Similarly, prior 
knowledge may not be used to answer implicit questions, as the answer must be related to a clue 
in the passage. Implicit questions are not included for the pre-primer passages, although the 
student “. . . may answer an explicit question by using information from the pictures” (p. 74). 
Five, six, or eight questions are provided for pre-primer through grade six passages. The 
following criteria are used to determine reading levels, based on the number of correct answers 



to questions: 
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Five questions: 


Independent level: 


5 correct 




Instructional level: 


4 correct 




Frustration level: 


0-3 correct 


Six questions: 


Independent level: 


6 correct 




Instructional level: 


4 correct 




Frustration level: 


0-3 correct 


Eight questions: 


Independent level: 


8 correct 




Instructional level: 


6-7 correct 




Frustration level: 


0-5 correct (p.74) 


Although total comprehension may be determined by calculating the percent of questions 



answered correctly for each passage, it is recommended that separate calculations be done for 
explicit and implicit questions. For example, the examiner should record the total number of 
explicit questions asked of the student and the total number answered correctly, and record the 
total number of implicit question asked of the student and the total number answered correctly; 
the percentage of correct answers for each category of questions may be obtained by dividing the 
total correct by the total asked and multiplying the result by 100. For the purposes of our study, 
comprehension scores for total comprehension and for category of questions (i.e., explicit and 
implicit comprehension) were used. 

The authors of the QRI-3 report reliability and validity information (Leslie & Caldwell, 
2001). Inter-scorer reliability, using three scorers who were reading teachers or specialists with 
master’s degrees, were obtained using Cronbach’s alpha (Cronbach, 1951) as follows: .99 for 
total miscues, .99 for meaning-change miscues, .98 for explicit comprehension, and .98 for 
implicit comprehension. Internal consistency reliability was calculated using the standard error 
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of measurement (SEM) procedure described by Crocker and Algina (1986) for criterion- 
referenced tests “where there is reduced variability in subject’s performance” (Leslie & 
Caldwell, 2001, p. 436). According to Crocker and Algina (1986), the standard error must be 
between .00 and .25/number of items minus one; the lower the result, the better the internal 
consistency. The SEMs of all passages (Pre-primer through high school) ranged from .12 to .21, 
all being within the acceptable SEM range. Alternate-form reliability for criterion-referenced 
tests was estimated using Livingston’s K formula; the index indicates the magnitude of the 
discrepancy of misclassification when judging the reliability of an instructional decision. Using 
two similar passage types (i.e., narrative or expository), if a student’s performance on both 
indicates the same instructional level, then there is evidence of alternate form reliability. Based 
on comprehension scores obtained from both passages the reliabilities of the instructional-level 
decisions were above .80; across readability levels, 71% to 84% of the time the same 
instructional level was found on both passages. Reliability of diagnostic profiles was examined 
by two judges who independently classified 108 readers’ abilities according to word recognition 
and comprehension; the judges agreed on the students’ abilities 87% of the time. Also, the 
findings of studies designed to assess the QRI’s sensitivity to change, as a result of instructional 
interventions, indicate that the instrument is sensitive to immediate change (Glass, 1989) and to 
long-term (3-7 months) interventions (Leslie & Allen, 1999; Regner, 1992). 

Content validity, criterion-related validity, and construct validity of the QRI-3 are 
discussed. Regarding content validity, Leslie and Caldwell (2001) state that they used the 
reading research literature as the basis for test development. Thus, the QRI-3 includes both 
narrative and expository material and passages vary in content familiarity; the instrument 
assesses prior knowledge, examines both total miscues and uncorrected miscues that change the 
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meaning of the passage, and includes three measures of comprehension (i.e., retelling, implicit 
questions, and explicit questions). 

Criterion-related validity was examined by correlating students’ QRI-3 instructional 
levels for familiar material with their NCE Total Reading Scores on standardized achievement 
tests, by grade level, for grades 1, 2, 3, 4, and 8. All of the correlations were statistically 
significant (p < .01 or p < .05). Also, Regner (1992) reported statistically significant correlations 
between weighted word-recognition scores on the QRI-3 and combined word-identification and 
word-attack scale scores on the Woodcock Reading Mastery Test-Revised ( WRMT-R , 1987) and 
between weighted QRI-3 comprehension scores with passage comprehension on the WRMT-R. 

Evidence of construct validity was found in the high intercorrelations among word 
identification, total oral reading accuracy, acceptable accuracy, and rate of reading for beginning 
readers. Also, high intercorrelations were obtained among conceptual-knowledge scores, 
prediction, retelling, and comprehension. Further evidence of construct validity was found using 
stepwise regression analyses with grade instructional levels, acceptable miscues, reading rate, 
prior conceptual knowledge, and type of text (i.e., narrative or expository) serving as the 
predictor variables and total comprehension serving as the dependent variable. The results were 
that for the group of children at the pre-primer, primer, and first grade instructional levels with 
poor word-identification skills, nothing predicted comprehension; for children at the same 
instructional levels who had good word-identification skills, acceptable miscues accounted for 
16% of the variance in oral reading comprehension and type of text explained an additional 7% 
of the variance in comprehension scores. For the group of children at the second and third grade 
instructional levels with poor word-identification skills, nothing predicted oral or silent reading 
comprehension; for children at the same instructional levels who had good word-identification 
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skills, type of text accounted for 26% of the variance in oral reading comprehension scores and 
for 16% of the variance in silent reading comprehension scores. For the group of children at the 
fourth, fifth, and sixth grade instructional levels who had good word identification skills, prior 
knowledge accounted for 34% of the variance in oral reading comprehension scores and for 23% 
of the variance in silent reading comprehension scores. Leslie and Caldwell (2001) concluded 
that at the lower reading instructional levels, comprehension is best predicted by the percentage 
of miscues that retain meaning and by type of text, while at the upper reading instructional levels 
the conceptual knowledge that the reader possesses prior to reading a text is the best predictor of 
comprehension. 

The Mississippi Curriculum Test ( MCT) assesses reading performance in seven 
categories: Context Clues (Semantic); Word Structure (Syntactic); Word Patterns (Phonetic 
Structure); Vocabulary; Main Idea and Details (Textual); Expanded Comprehension 
(Metacognitive); and Workplace Data (Evaluative). MCT reliability estimates using Cronbach’s 
coefficient alpha ranged from .88 to .90 ( Mississippi Curriculum Test, 2003). In our study, data 
from six of the seven MCT categories that compose the reading subtest were used because the 
Workplace Data (Evaluative) category is not assessed until Grade 4. 

Procedures 

RM Instruction . In 2000, the school implemented in grades K-6 a sequential, systematic, 
direct instruction reading program, Reading Mastery, Rainbow Edition, developed by 
Engelmann and Hammer (1995), in an effort to improve elementary students’ reading 
performance. Reading Mastery (RM) includes: (a) explicit instruction in phonemic awareness, 
(b) direct, systematic teaching of phonics skills, (c) direct teaching and practice in developing 
fluency, (d) direct teaching of vocabulary, and (e) direct teaching of comprehension strategies 
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and skills (“Reading and Learning,” 2001). 

RM requires a minimum of 90 minutes of direct reading instruction per student per day, 
which includes a minimum of 45 minutes of direct instruction in phonics followed by a 
minimum of 45 minutes of group oral reading and vocabulary drills. RM contains highly- 
scripted lessons offered to small, homogeneously-grouped students with each lesson focused on 
a well-defined set of skills, followed by independent and small group activities so that students 
can practice and generalize skills; thus, in practice, every student receives approximately 120 
minutes or more of direct instruction in reading and language arts activities per day. The format 
of RM lessons requires a large portion of classroom time devoted to fast-paced, teacher-directed 
instruction that is “. . . punctuated by rhythmic choral-group and individual student responses” 
(Master Learners, 2002, p. 6). The goal is for all students to reach 100 percent mastery, thus “. . 
. teachers may ask 300 or more questions in six small-group sessions each day and perform 
reading checks at frequent intervals” Master Learners, 2002, p. 6). 

Also, RM requires intensive training for all teachers, so that they will understand the 
script that must be followed to effectively implement the program. RM requires an initial 15-25 
hours of teacher training followed by on-going training; the teachers who have been with the 
school district since 2000 have received over 400 hours of training. Regularly conducted 
classroom observations of teachers ensure fidelity to the implementation of direct instruction 
procedures. 

Data Collection . The school superintendent invited a research team from a nearby 
university to assess the impact of RM on the participants’ reading performance. The research 
team administered the HGLP and the QRI-3 in April 2001 when the participants had received 
one school year of RM, referred to as Year 1, and administered the instruments again in April of 
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the four subsequent years as the participants progressed through the grades, referred to as Year 2, 
Year 3, Year 4, and Year 5, respectively. The HGLP and the QRI - 3 were administered by 
graduate students in literacy education who had been trained to administer the measures. The 
HGLP was administered to the participants at each school in small groups, by grade level and by 
classroom (i.e., when two or more participants were in the same classroom), over a period of six 
days (i.e., a day of testing per school was required to complete testing of all participants). The 
QRI-3 was administered to the participants, individually, using the oral and silent reading 
passages that corresponded to the grade level of each participant. Kindergarten students 
responded only to the oral narrative kindergarten-level passage of the ORI-3\ students in grades 
one and above responded to two grade-level narrative passages (oral and silent reading forms) 
and the grade-level expository passage (silent reading form). An additional source of data was 
the participants’ reading scale scores from the MCT (a measure used by the state to assess 
general reading performance) which were obtained annually from the school district. 

Analysis 

Given that all elementary schools in the school system had implemented RM , a straight 
forward analysis and interpretation of the effects of the RM program was not possible since no 
control or comparison group was available. In an attempt to overcome this limitation, the 
available data were analyzed in three parts. First, the data for those participants who began the 
RM program in kindergarten was examined descriptively and graphically. Second, statistical 
comparisons were made by comparing the performance of those participants who began the RM 
program in kindergarten with those participants who began the program at some later grade. For 
example, the performance of (a) those participants who began RM in kindergarten and completed 
grade one was compared with that of participants who did not begin the RM program until 
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grade 1; (b) those participants who began RM in kindergarten and completed grade two were 
compared to that of participants who did not begin the RM program until grade two; and (c) 
those participants who began RM in kindergarten and completed grade three were compared with 
that of participants who did not begin the RM program until grade three. Third, performance of 
those participants who began the RM program in kindergarten was examined descriptively and 
graphically by comparing their performance with that of all other participants available at each 
year of the program. 

Results 

Data Analysis, Part 1 

The results for the first set of analyses are presented in Tables 1 through 1 1 and Figures 2 
through 23. For each of the reading performance variables examined, two figures are provided to 
show the performance of students across the five years of the program for those who began RM 
in kindergarten. The first of each pair of figures shows the performance of each of the five 
kindergarten groups as intact groups as they progressed through subsequent grades. The second 
figure of each pair shows the performance by grade-level groups (by combining all K data, all 
grade 1 data, etc.), regardless of the length of time that students had received RM instruction. 
Since the performance of the groups (i.e., intact groups and grade-level groups) was similar 
across the years, as can be seen in the first of each pair of figures, the second figure appears to 
more clearly describe the performance of students across all five years. 

MCT Reading scale score performance for students who began the RM program in 
kindergarten increased linearly across grades 2, 3, and 4 (Figures 2 and 3). Average performance 
for these students was above the cut-off for proficient performance for each of the three grade 



levels. 
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On the HGLP (CLOZE) measure, student performance tended to improve across grades 
1, 2, and 3 and then decreased at grade four (Figure 5). For grades 1, 2, and 3, average student 
performance on the HGLP (CFOZE) measure was at or above the independent reading level. 

Examination of Figure 7 indicated that the percentage of students performing at the 
independent oral reading level was modest at best, reaching a maximum of about 36% at grade 
three. Also, the percentage of students reading at the instructional level or higher on the QRI-3 
oral reading measure increased through year two and then appeared to stabilize at about 70% 
(Figure 9). The percentage of students reading below the frustration level on the QRI-3 oral 
reading measure dropped notably through grade 2 but then appeared to stabilize at around 25% 
(Figure 11). 

Students’ silent reading performance (expository text) was quite poor. Very few students 
were reading at the independent or instructional level for grades K through 4 (Figures 12 through 
15) and the percentage reading at the frustration level was consistently above 90% (Figure 17). 

Explicit, implicit and total reading comprehension was consistently low rarely exceeding 
the 30% correct level (Figures 18 through 23). 

Data Analysis, Part 2 

The results of the statistical analyses constituting part 2 are presented in Tables 12 and 
13. Each of the comparisons presented in Tables 12 and 13 are such that the students beginning 
RM in kindergarten are compared with those students who began RM at a later grade level. 

A statistically significant difference was found in the MCT Reading scale score means of 
those students who began RM in kindergarten and those who began RM at grade 2 ( t(266 ) = 2.83, 
p < .01) with the mean of those who began RM in kindergarten exceeding that of those who 
began RM at grade 2 (Table 12). However, no statistically significant difference in the means of 
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those students who began RM in kindergarten and those who began RM in grade 3 was found 
(t(225) = 1.92, ns). 

The only statistically significant difference found for the HGLP (CLOZE) measures was 
at the end of grade 1 (t(418) = 5.05, p < .01). The mean CLOZE percentage score was greater for 
those who began RM in kindergarten (58.44) than for those who began RM at grade 1 (45.21). 

No statistically significant differences were found at grades two and three (Table 12). 

For the QRI-3 oral and silent reading measures, no statistically significant differences in 
the percentage of students performing at the instructional level or higher were found for grades 1, 
2, and 3 (Table 13). Comparison of the explicit, implicit, and total comprehension levels yielded 
statistically significant differences ip < .01) only on the implicit comprehension measures at 
grades 1 and 2. However, at grade 1 those students who did not begin RM in kindergarten out 
scored the children who began RM in kindergarten. The reverse held at grade 2 with those 
students who began RM in kindergarten scoring higher. 

Data Analysis, Part 3 

Part three of the analysis compared the performance of those who were in the RM 
program beginning in kindergarten with all students who began the program at a later grade 
level. The results are presented descriptively in Tables 14 through 20 and graphically in Figures 
24 through 45. For each pair of these figures, the first figure indicates that the performance of 
those who began the RM program in kindergarten is virtually identical to that of those who began 
the program at later grades. The second figure of each pair presents the performance averaged 
across all groups. Little difference in performance between those who began RM in kindergarten 
and those who began RM at a later grade was found on any of the measures examined. 
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In summary, it was found that: (a) MCT mean scores increased linearly from grades 2 to 
7 ; (b) CLOZE performance increased from grades 1 to 3 and decreased thereafter; (c) the 
percentages of students performing at the independent reading level on narrative texts (oral 
reading) were low, generally in the 30s; (d) the percentages of students performing at the 
instructional reading level on narrative texts (oral reading) were generally in the 80s; (e) the 
percentages of students performing at the frustration reading level on narrative texts (oral 
reading) were generally in the 20s; (f) virtually no students performed at the independent 
reading level on expository texts (silent reading); (g) there was some improvement in 
percentage of students performing at the instructional reading level on expository texts (silent 
reading) beginning at grade 5; (h) percentages of students performing at the frustration reading 
level on expository texts (silent reading) were generally in the upper 70s; and (i) explicit, 
implicit, and total reading comprehension scores indicated slow improvement across the years. 

Discussion 

The design of this longitudinal study required that grade-level passages be administered 
annually, using both oral and silent reading assessments, so that the extent to which RM 
instruction promoted grade-level or higher reading comprehension could be examined over a 
five-year period of time. Overall, the results of this study suggested that sequential, systematic 
direct instruction via the RM program over the five-year period did not have a consistent positive 
impact on reading comprehension scores of students. This finding was supported by three 
procedures used to examine the data: descriptive results, statistical comparisons, and graphs 
comparing group performance over the five years. 

The first set of descriptive analyses (Tables 1-11 and Figures 2-23) contrasted grade- 
level performance of students who began RM in kindergarten with groups of students who began 
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RM at later designated grade levels; for example, in year 5 of the study students who began RM 
in kindergarten were in grade 4, having received only RM instruction throughout the five-year 
period, so this group's performance at each grade level served as a point of comparison for the 
corresponding grade-level performance of groups of students who began RM after the 
kindergarten year. In general, the performance of the students was such that: (a) students who 
began RM in kindergarten performed similarly to students who began RM after the kindergarten 
year; (b) the means of the MCT reading scale scores fell above the “proficient” level and 
increased linearly over the years examined; (c) mean scores on the HGLP (CLOZE) measure 
changed in a clearly nonlinear fashion with students performing at or above the “independent” 
level for grades K through 3 and dropping thereafter; (d) student performance on the oral reading 
measure was such that generally less than one third of the students were reading at the 
independent level over the years examined, but the percentage of students reading at the 
instructional level increased over the years leveling off at about 80% with a commensurate 
decrease in the percentage of students reading at the frustration level; (e) the performance of the 
students on the silent reading tasks was disappointing with very few students reading at the 
independent level and nearly all reading at the frustration level; and (f) on the explicit, implicit, 
and total reading comprehension measures, performance was generally low over the years 
examined though modest improvement was indicated over the years. These results contradict the 
findings of Meyer, Gersten, and Gutkin (1983) who reported that Follow Through students who 
received sequential, systematic direct instruction via DISTAR in kindergarten through third 
grades maintained significantly higher reading scores than comparison students in grades 4 and 
5. 



Of particular interest is the contrast in student performance on the MCT and the cloze 
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measure where the MCT scale scores improved linearly over the grade levels while the cloze 
scores tended to level off or drop after grade 3, a finding further substantiated in part three of the 
analysis. The MCT places considerable emphasis on assessment of word recognition skills, such 
as word structures, word patterns, phonics, and vocabulary, with comprehension assessment 
focusing on literal comprehension with some attention to inferential comprehension. In contrast 
to the MCT , both the HGLP (CLOZE) measure assesses a fuller range of reading comprehension 
skills (e.g., literal, inferential, evaluative, etc.). Some caution should be exercised in the 
interpretation of this drop, however, since the mean at grade 4 is based on the smallest number of 
students. It is noted, however, that HGLP (CLOZE) scores dropped at the fourth-grade level as 
well for students who did not enter the RM program in kindergarten (see Figures 26 and 27). 
More research is needed to explore why initial gains in reading comprehension scores may, or 
may not be, sustained in subsequent years of RM instruction. Collectively, the results of the 
descriptive analyses provide little support for RM. 

Results of the statistical analyses (Tables 12 and 13), further indicated lack of a 
consistent, positive impact by RM on students’ reading comprehension performance. Of the 
twenty statistical comparisons made between students who began RM in kindergarten with 
students beginning RM at some later grade, only four were statistically significant. Students 
beginning RM in kindergarten had higher mean scores on the MCT and cloze at grade 1 and a 
higher mean implicit reading comprehension scores at grade two than did students beginning RM 
at those grade levels (/? < .01). However, the students who began RM at grade 2 out performed (p 
< .01) the students who began RM in kindergarten at grade 2. These results provide little support 



for the RM program. 
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Results of the third analysis of data reinforced the findings of the statistical analyses, as 
shown in the 11 pairs of graphs (Figures 24-45) and the associated tables (Tables 14-20) from the 
analyses. Over the years examined, the performance of the students on each of the measures, as 
seen in the first of each pair of graphs, is remarkably similar regardless of when they began RM. 
Student performance on the MCT continued to improve linearly, cloze performance tended to 
drop and level off after grade 3, performance on the oral reading tasks was clearly better than 
that for the silent reading tasks, and comprehension on the explicit, implicit and total reading 
comprehension measures of the QRI tended to improve slowly. 

It was disappointing to see that during the five-year period of study the percentage of 
students reading at the frustration reading level QRI narrative passages was so high particularly 
on the QRI expository passages, especially when considering that all passages were grade-level 
selections. Even though reading experts (Dreher & Sammons, 1994; Duke, 2004; Frey & Fisher, 
2007) recognize that expository reading tends to be more difficult for students, it was 
disconcerting to find such high percentages of students unable to comprehend grade-level 
expository texts. Some studies have reported that lack of prior knowledge about a topic 
decreases comprehension Marr & Gormley, 1982; Pearson, Hansen, & Gordon, 1979; Taft and 
Feslie, 1985), so this may have been a factor in students' poor comprehension of expository 
passages. 

Collectively, results of the three analyses of this longitudinal data indicate that neither the 
grade level at which students began the RM program nor the length of time that students 
participated in the RM program impacted reading comprehension performance in a consistently 
positive direction. The credibility of this interpretation of the results of the study is supported by 
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the fact that the students were randomly drawn from six different schools and the study was done 
by researchers who were not officially connected with the school district. 

Since the RM program requires that all students, regardless of their grade levels, progress 
through the same sequence of lessons, the focus on graphophonemic skills in the early lessons 
may have affected students' comprehension performance; over-reliance on the graphophonemic 
cue system indicates an imbalance in the reader's use of the three cue systems during the reading 
process, and an imbalance tends to have a negative affect on reading comprehension (Burke, 
1976; Christie, 1981; Menosky, 1976; Taft & Leslie, 1985). According to Allen (1976), “. . . 
growth in reading is retarded when individuals are asked to distort their language in drills which 
isolate the phonemic and graphic systems from functioning language contexts” (p. 73). 

Although a few statistically significant differences were found in part 2 of the analysis, 
the preponderance of data provide little evidence to support the viability of the RM program for 
the development of reading comprehension abilities of students as they progress through the 
elementary grades. It should be kept in mind that the absence of a control group makes these 
conclusions less than definitive. Nevertheless, based on the data available, little support for the 
program was found. 

The findings of this study raise some questions to be addressed in future research 
studies. First, will students who receive only RM instruction for three or more consecutive 
school years, beginning in kindergarten, be able to read grade-level narrative and expository 
texts with adequate comprehension (i.e., 75% or higher) as they progress through the grades. 

Second, what is the optimum time period (i.e., grade levels) in which RM instruction 
benefits oral- and silent-reading comprehension performance? And, will continued sequential, 
systematic direct instruction beyond the optimum time period have a positive, neutral, or 
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negative impact on reading performance? The findings in this study suggest that the benefits of 
RM instruction are more evident from kindergarten through grade two, but further research is 
needed to answer this question. 

Third, is there a relationship between RM instruction and reading comprehension 
demands associated with increase in grade level? If so, is the relationship a positive or a 
negative one? For example, if the major reading comprehension demand in the lower grades 
tends to be at the explicit level, do we find that sequential, systematic direct instruction is more 
effective than when reading comprehension demands are at implicit levels, such as the 
interpretive or evaluative levels 

Fourth, do students who exhibit low, average, or high early reading ability benefit in 
similar ways from RM instruction? Scarcelli (1999) compared the effects of direct instruction 
and whole language instruction on first-grade students’ decoding and comprehension 
performance and reported that students in the average to below average range (i.e., the bottom 
third) of reading ability did significantly better with direct instruction, but that type of instruction 
did not produce significant post-test differences for students in the middle-third and upper-third 
subgroups. Scarcelli noted, however, that students in the top-third subgroup (based on pretest 
results) who entered first grade with a strong vocabulary and good decoding skills seemed to 
achieve and progress at a quicker rate in the whole language environment. More research is 
needed to determine how RM impacts different levels of early reading ability. 

Since the findings of this study may provide useful information to school administrators 
who are considering the implementation of RM (or some other DI program), the following 
suggestions are offered. First, the inclusion of a comparison group of randomly selected students 
will allow for a comparison of the effects of RM instruction and some other instructional 
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approach to reading. Second, a longitudinal study designed to address some of the questions 
raised in this study may provide insights into the specific ways in which a selected DI program 
impacts reading performance. Third, the initial implementation of a DI program should occur at 
the kindergarten level, as the impact of DI on the reading performance of students who have 
been exposed to other instructional programs is unclear, especially when DI begins after second 
grade. Based on our study, the greatest benefits of RM occurred from kindergarten through 
grade two, especially between grades 1 and 2. Four, the inclusion of both expository and 
narrative texts as measures of comprehension is recommended, since expository text is used 
increasingly as students progress through the grades. Five, the selection of a DI program which 
balances instruction in word-recognition and reading-comprehension strategies is recommended. 
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Figure 1. Grade-Level Composition for Eight Groups of Participants Across the 



Five-Year Period of RM Instruction 
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Figure 2. MCT Reading Scale Score Performance for Students Beginning RM in Kindergarten by Year 
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Figure 3. MCT Reading Scale Score Performance for Students Beginning RM in Kindergarten by Year 
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MCT Reading Scale Scores for Students Beginning RM in Kindergarten by Program Year 
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Figure 4. Mean HGLP ( CLOZE ) Performance (%) for Students Beginning RM in Kindergarten 
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Figure 5. Mean FIGLP (CLOZE) Performance (%) for All Students 
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HGLP ( CLOZE ) Performance of Students Beginning RM in Kindergarten by Program Year 
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Figure 6. 



Percent of Students Reading at the Independent Level for Students Beginning RM in Kindergarten 




Grade 

Figure 7. Percent of Students Reading at the Independent Level for All Students 
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Percent of Students Beginning RM in Kindergarten Reading at the Independent Level — Narrative (Oral) Reading 
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Figure 8. Percent of Students Reading at the Instructional Level (Independent or Instructional) for Students Beginning RM in Kindergarten 
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Figure 9. Percent of Students Reading at the Instructional Level (Independent or Instructional) for All Students 
Table 4 

Percent of Students Beginning RM in Kindergarten Reading at the Instructional (Independent or Instructional) Level — Narrative (Oral) Reading 
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42.86 


40 


63 


63.49 


50 


69 


72.46 


44 


59 


74.58 








167 


268 


62.31 


2 


48 


69 


69.57 


48 


56 


85.71 


43 


55 


78.18 














139 


180 


77.22 


3 


44 


55 


80.00 


39 


48 


81.25 




















83 


103 


80.58 


4 


33 


48 


68.75 


























33 


48 


68.75 
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Figure 10. Percent of Students Reading at the Frustration Level for Students Beginning RM in Kindergarten 
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Figure 11. Percent of Students Reading at the Frustration Level for All Students 



Table 5 

Percent of Students Beginning RM in Kindergarten Reading at the Frustration) Level — Narrative (Oral) Reading 







Year 1 






Year 2 






Year 3 






Year 4 




Year 5 






All 




Grade 


n 


N 


P 


N 


N 


P 


n 


N 


P 


n 


N 


P 


n N 


P 


n 


N 


P 


K 


80 


93 


86.02 


64 


72 


88.89 


79 


85 


92.94 


81 


94 


86.17 


83 89 


93.26 


387 


433 


89.38 


1 


44 


77 


57.14 


23 


63 


36.51 


19 


69 


27.54 


15 


59 


25.42 






101 


268 


37.69 


2 


21 


69 


30.43 


8 


56 


14.29 


12 


55 


21.82 












41 


180 


22.78 


3 


11 


55 


20.00 


9 


48 


18.75 


















20 


103 


19.42 


4 


15 


48 


31.25 
























15 


48 


31.25 
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Figure 12. Percent of Students Reading at the Independent Level for Students Beginning RM in Kindergarten 
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Figure 13. Percent of Students Reading at the Independent Level for All Students 



Table 6 

Percent of Students Beginning RM in Kindergarten Reading at the Independent Level — Expository (Silent) Reading 







Year 1 






Year 2 






Year 3 






Year 4 




Year 5 






All 




Grade 


n 


N 


P 


n 


N 


P 


n 


N 


P 


n 


N 


P 


n N 


P 


n 


N 


P 


K 


0 


95 


0.00 


0 


73 


0.00 


0 


84 


0.00 


2 


87 


2.29 


0 86 


0.00 


2 


425 


0.47 


1 


0 


74 


0.00 


0 


63 


0.00 


2 


68 


2.94 


0 


58 


0.00 






2 


263 


0.76 


2 


0 


69 


0.00 


0 


56 


0.00 


2 


55 


3.63 












2 


180 


1.11 


3 


0 


56 


0.00 


0 


47 


0.00 


















0 


103 


0.00 


4 


0 


47 


0.00 
























0 


47 


0.00 
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Figure 14. Percent of Students Reading at the Instructional Level (Instructional or Independent) for Students Beginning RM in Kindergarten 
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Figure 15. Percent of Students Reading at the Instructional Level (Independent or Instructional) for All Students 



Table 7 

Percent of Students Beginning RM in Kindergarten Reading at the Instructional (Independent and Instructional) Level — Expository (Silent) 
Reading 







Year 1 






Year 2 






Year 3 






Year 4 






Year 5 






All 




Grade 


n 


N 


P 


n 


N 


P 


n 


N 


P 


n 


N 


P 


n 


N 


P 


n 


N 


P 


K 


2 


95 


2.11 


0 


73 


0.00 


i 


84 


1.19 


7 


94 


7.45 


6 


86 


6.98 


16 


432 


3.70 


1 


0 


74 


0.00 


1 


63 


1.58 


8 


68 


11.76 


4 


58 


6.90 








13 


263 


4.94 


2 


0 


69 


0.00 


0 


56 


0.00 


2 


55 


3.64 














2 


180 


1.11 


3 


0 


56 


0.00 


0 


47 


0.00 




















0 


103 


0.00 


4 


0 


47 


0.00 


























0 


47 


0.00 
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Figure 16. Percent of Students Reading at the Frustration Level for Students Beginning RM in Kindergarten 
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Figure 1 7. Percent of Students Reading at the Frustration Level for All Students 



Table 8. 

Percent of Students Beginning RM in Kindergarten Reading at the Frustration) Level — Expository (Silent) Reading 







Year 1 






Year 2 




Year 3 




Year 4 


Year 5 






All 




Grade 


n 


N 


P 


n 


N 


P 


n 


N 


P 


n 


N 


P 


n N 


P 


n 


N 


P 


K 


93 


95 


97.89 


73 


73 


100.00 


83 


84 


98.81 


87 


94 


92.55 


80 86 


93.02 


416 


432 


96.30 


1 


74 


74 


100.00 


62 


63 


98.41 


60 


68 


88.24 


54 


58 


93.10 






250 


263 


95.06 


2 


66 


69 


95.65 


53 


56 


94.64 


50 


55 


90.91 












169 


180 


93.89 


3 


51 


56 


91.07 


47 


47 


100.00 


















98 


103 


95.15 


4 


46 


47 


97.87 
























46 


47 


97.87 
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Figure 18. Explicit Reading Comprehension Performance (Percent Correct) for Students Beginning RM in Kindergarten 




Figure 19. Explicit Reading Comprehension Performance (Percent Correct) for All Students 



Table 9 



Explicit Reading Comprehension Performance (Percent Correct) for Students Beginning RM in Kindergarten 







Year 1 






Year 2 






Year 3 






Year 4 






Year 5 






All 




Grad 

e 


M 


s 


n 


M 


S 


n 


M 


S 


n 


M 


S 


n 


M 


s 


n 


M 


S 


n 


K 


13.6 


18.0 


9 


11.4 


16.6 


7 


10.7 


15.1 


8 


17.9 


22.3 


9 


23.1 


22.8 


8 


15.5 


19.6 


42 




1 


4 


0 


9 


5 


4 


1 


7 


4 


5 


4 


4 


1 


1 


6 


4 


2 


8 


1 




15.0 


7 


11.9 


17.4 


6 


22.9 


29.1 


6 


20.6 


24.4 


5 








15.9 


22.0 


26 




9.17 


1 


5 


0 


5 


3 


8 


8 


8 


9 


0 


8 








1 


2 


4 


2 


23.0 


27.3 


6 


21.8 


26.5 


5 


34.5 


31.6 


5 














26.2 


28.5 


17 




8 


5 


5 


8 


7 


6 


5 


4 


5 














8 


2 


6 


3 


21.6 


12.5 


5 


26.6 


21.5 


4 




















23.9 


17.4 






3 


0 


2 


0 


9 


7 




















9 


1 


99 


4 


39.8 


28.5 


4 


























39.8 


28.5 






3 


1 


3 


























3 


1 


43 
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Figure 20. Implicit Reading Comprehension (Percent Correct) for Students Beginning RM in Kindergarten 




Figure 21. Implicit Reading Comprehension (Percent Correct) for All Students 



Table 10 



Implicit Reading Comprehension Performance (Percent Correct) for Students Beginning RM in Kindergarten 







Year 1 






Year 2 






Year 3 






Year 4 






Year 5 






All 




Grad 

e 


M 


s 


n 


M 


S 


n 


M 


S 


n 


M 


S 


ft 


M 


s 


ft 


M 


S 


ft 


K 


18.0 


25.9 


9 


15.2 


26.7 


7 


11.0 


18.3 


8 


15.1 


25.4 


9 


18.3 


25.2 


8 


15.6 


24.5 


42 




6 


7 


0 


0 


6 


4 


1 


5 


4 


6 


6 


4 


1 


7 


6 


0 


7 


8 


1 


16.0 


24.8 


7 


13.1 


25.3 


6 


27.5 


32.6 


6 


26.7 


26.8 


5 








20.6 


27.6 


26 




0 


8 


5 


0 


4 


3 


7 


5 


8 


2 


5 


8 








4 


1 


4 


2 


19.0 


19.5 


6 


29.6 


26.3 


5 


36.1 


22.9 


5 














27.7 


22.9 


17 




4 


2 


5 


9 


9 


6 


4 


1 


5 














7 


4 


6 


3 


42.5 


26.1 


5 


38.5 


20.9 


4 




















40.6 


23.5 






5 


1 


2 


6 


9 


7 




















6 


3 


99 


4 


28.7 


25.3 


4 


























28.7 


25.3 






8 


7 


7 


























8 


7 


43 
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Figure 22. Total Comprehension (Explicit and Implicit) Percent Correct for Students Beginning RM in Kindergarten 




Figure 23. Total Comprehension (Explicit and Implicit) Percent Correct for All Students 



Table 11 



Total Reading Comprehension Performance (Explicit and Implicit) Percent Correct for Students Beginning RM in Kindergarten 







Year 1 






Year 2 






Year 3 






Year 4 






Year 5 






All 




Grad 

e 


M 


S 


n 


M 


S 


n 


M 


S 


n 


M 


S 


n 


M 


S 


ft 


M 


5 


ft 


K 


15.8 


22.01 


9 


13.3 


21.7 


7 


10.8 


16.7 


8 


16.5 


23.9 


9 


23.1 


18.7 


8 


16.0 


20.6 


42 




3 




0 


4 


0 


4 


6 


7 


4 


6 


0 


4 


1 


5 


6 


5 


3 


8 


1 


12.5 


19.95 


7 


12.5 


21.4 


6 


25.2 


30.9 


6 


23.7 


25.6 


5 








18.2 


24.4 


26 




8 




5 


0 


0 


3 


8 


2 


8 


1 


3 


8 








8 


8 


4 


2 


21.0 


23.44 


6 


25.7 


26.4 


5 


35.3 


27.2 


5 














27.0 


25.7 


17 




6 




5 


8 


8 


6 


4 


8 


5 














2 


7 


6 


3 


32.0 


26.94 


5 


32.5 


21.2 


4 




















32.3 


24.1 






9 




2 


8 


9 


7 




















2 


2 


99 


4 


34.3 


26..9 


4 


























34.3 


26.9 






0 


4 


3 


























0 


4 


43 
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Table 12 

Comparisons of Means on MCT and HGLP CLOZE Measures: Students Beginning RM in Kindergarten Compared with Students Beginning RM 
at the Designated Grade 



When RM Began 








At Kindergarten 




At Designated Grade 










Measure 


Grade 


Mean 


s 


n 


Mean 


5 


n 


t 


df 


P 


MCT 


2 


448.69 


41.03 


171 


432.04 


54.34 


97 


2.83 


266 


<.01 




3 


464.36 


41.63 


140 


452.71 


48.72 


87 


1.92 


225 


ns 


CLOZE 


1 


58.44 


19.05 


290 


45.21 


34.43 


130 


5.05 


418 


<.01 




2 


62.12 


19.40 


196 


59.36 


24.59 


131 


1.13 


325 


ns 




3 


63.36 


15.98 


102 


63.63 


15.79 


131 


-0.13 


231 


ns 


Explicit 


1 


15.91 


22.05 


264 


12.23 


19.99 


94 


1.42 


356 


ns 




2 


26.28 


26.78 


176 


26.35 


24.81 


102 


-0.02 


276 


ns 




3 


23.99 


22.79 


99 


23.27 


21.40 


65 


0.20 


164 


ns 


Implicit 


1 


20.64 


27.61 


264 


31.90 


33.28 


94 


-3.21 


356 


<.01 




2 


27.77 


22.94 


176 


19.85 


20.46 


102 


2.88 


276 


<.01 




3 


40.66 


25.53 


99 


38.08 


21.47 


65 


0.68 


164 


ns 


Total 


1 


18.28 


24.48 


264 


18.79 


19.97 


94 


-0.18 


356 


ns 




2 


27.02 


25.77 


176 


23.10 


18.63 


102 


1.35 


276 


ns 




3 


32.32 


24.12 


99 


30.67 


17.32 


65 


0.48 


164 


ns 



Table 13 

Comparisons of Percentages of Students Performing at Instructional or Higher Reading Levels on the QRI: Students Beginning RM in 
Kindergarten Compared with Students Beginning RM at the Designated Grade 



When RM Began 








At Kindergarten 


At Designated Grade 






Measure 


Grade 


n 


p 


n 


P 


Z 


P 


Oral Reading 


i 


263 


62.31 


103 


64.10 


-0.32 


ns 




2 


180 


77.22 


108 


80.40 


-0.59 


ns 




3 


103 


80.58 


77 


89.60 


-1.65 


ns 


Silent Reading 


1 


263 


4.94 


110 


3.60 


0.57 


ns 




2 


180 


1.11 


85 


0.00 


0.97 


ns 




3 


103 


0.00 


99 


0.00 


0.00 


ns 
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Table 14 

Summary Data by Grade Level — MCT Reading Scale Scores 



Beginning Grade of RM 


Grade 




K 


1 


2 


3 


Combined 


2 


Mean 


448.69 


446.41 


432.04 




443.52 




S.D. 


41.03 


46.20 


54.34 




49.32 




N 


171 


80 


97 




348 


3 


Mean 


464.36 


459.45 


458.06 


452.71 


459.37 




S.D. 


41.63 


45.67 


49.92 


48.72 


46.56 




N 


140 


84 


95 


87 


406 


4 


Mean 


492.35 


489.19 


467.27 


484.70 


479.42 




S.D. 


52.21 


44.64 


48.42 


51.00 


48.34 




N 


40 


81 


107 


74 


302 


5 


Mean 




509.74 


505.55 


500.36 


504.86 




S.D. 




55.26 


48.72 


50.68 


51.68 




N 




69 


87 


88 


244 


6 


Mean 






513.37 


514.68 


514.04 




S.D. 






53.18 


37.85 


34.76 




N 






73 


77 


150 


7 


Mean 








525.15 


528.15 




S.D. 








38.85 


38.85 




N 








67 


67 
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Table 15. 

Summary Data by Grade Level — HGLP (CLOZE), Percent 



Beginning Grade of RM 


Grade 




K 


1 


2 


3 


Combined 


i 


Mean 


58.44 


45.21 






54.35 




S.D. 


19.05 


34.43 






32.74 




N 


290 


130 






420 


2 


Mean 


62.12 


65.99 


59.36 




62.03 




S.D. 


19.40 


17.86 


24.59 




25.16 




N 


196 


84 


131 




422 


3 


Mean 


63.36 


63.35 


63.43 


63.63 


63.45 




S.D. 


15.98 


17.23 


17.85 


15.79 


21.80 




N 


102 


103 


108 


131 


444 


4 


Mean 


51.07 


55.35 


55.65 


54.30 


54.61 




S.D. 


12.76 


13.87 


16.18 


14.62 


19.01 




N 


42 


81 


95 


85 


319 


5 


Mean 




58.57 


57.56 


52.19 


55.67 




S.D. 




15.39 


16.24 


14.85 


20.09 




N 




61 


87 


98 


246 


6 


Mean 






47.68 


48.11 


47.90 




S.D. 






15.53 


17.17 


19.60 




N 






73 


75 


148 


7 


Mean 








36.27 


36.27 




S.D. 








8.41 


8.41 




N 








73 


73 





Percent Reading at Indep. Level-Oral ^ Percent Reading at Indep. Level-Oral 
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ure 28. Percent of Students Reading at the Independent Level by Grade RM began — Narrative (Oral) Reading 




Figure 29. Percent of Students Reading at the Independent Level, Combined All Students by Grade Level — Narrative (Oral) Reading 








Grade 



Figure 31. Percent of Students Reading at the Instructional Level Combined All Students by Grade Level — Narrative (Oral) Reading (Includes 



Students Performing at either the Instructional or the Independent Reading Level) 
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Table 16. 



Percent of Students Reading at the Instructional, Independent, and Frustration Levels by Grade Level — Narrative (Oral) Reading 



Beginning Grade Level 


Grade 


Level 


K 


1 


2 


3 


Combined 


K 


Independent 


3.0 








3.0 




Instructional 


10.6 








11.2 




Frustration 


89.4 








88.3 


1 


Independent 


17.5 


33.0 






21.8 




Instructional 


62.3 


64.1 






62.8 




Frustration 


37.7 


35.9 






37.2 


2 


Independent 


35.6 


36.0 


31.8 




34.6 




Instructional 


77.2 


76.4 


80.4 




77.9 




Frustration 


22.8 


23.6 


19.6 




22.1 


3 


Independent 


28.2 


12.2 


31.0 


57.1 


30.0 




Instructional 


80.6 


70.4 


83.0 


89.6 


80.4 




Frustration 


19.4 


29.6 


17.0 


10.4 


19.6 


4 


Independent 


12.5 


18.3 


13.3 


26.2 


17.7 




Instructional 


68.8 


72.0 


68.1 


77.4 


71.6 




Frustration 


31.3 


28.0 


31.9 


22.6 


28.5 


5 


Independent 




26.2 


35.2 


19.6 


27.0 




Instructional 




83.1 


80.7 


73.9 


78.8 




Frustration 




16.9 


19.3 


26.1 


21.2 


6 


Independent 






28.0 


22.4 


25.0 




Instructional 






85.3 


83.5 


89.7 




Frustration 






14.7 


16.5 


15.7 


7 


Independent 








41.7 


41.7 




Instructional 








90.3 


90.3 




Frustration 








9.7 


9.7 




Percent Reading at the Indep. Level-Silent ^ Percent Reading at the Indep. Level-Silent 
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Grade 




lire 34. Percent of Students Reading at the Independent Level by Grade RM began — Expository (Silent) Reading 




Figure 35. Percent of Students Reading at the Independent Level Combined All Students by Grade Level — Expository (Silent) Reading 
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Grade 




Figure 36. Percent of Students Reading at the Instructional Level by Grade RM began — Expository (Silent) Reading (Includes Students who 
Performed at either an Instructional or an Independent Reading Level) 



100 




Grade 

Figure 37. Percent of Students Reading at the Instructional Level Combined across Grade Level — Expository (Silent) Reading (Includes Students 
Performing at either the Instructional or the Independent Reading Level) 








Percent Reading at the Frustration Level-Silent ^ Percent Reading at the Frustration Level-Silent 
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ure 38. Percent of Students Reading at the Frustration Level, by Grade RM began — Expository (Silent) Reading 




Figure 39. Percent of Students Reading at the Frustration Level, Combined All Students by Grade Level — Expository (Silent) Reading 
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Table 17. 

Percent of Students Reading at the Instructional, Independent, and Frustration Reading Levels — Expository (Silent) Reading 



Beginning Grade of RM 


Grade 


Level 


K 


1 


2 


3 


Combined 


K 


Independent 


0.5 








0.5 




Instructional 


3.7 








3.7 




Frustration 


96.3 








96.3 


1 


Independent 


0.8 


1.0 






0.8 




Instructional 


4.9 


3.6 






4.5 




Frustration 


95.1 


96.4 






95.5 


2 


Independent 


1.1 


0.0 


1.7 




0.5 




Instructional 


1.1 


0.0 


2.5 




1.3 




Frustration 


93.9 


100.0 


97.5 




96.4 


3 


Independent 


0.0 


0.0 


0.0 


0.0 


0.0 




Instructional 


0.0 


0.0 


2.0 


1.3 


0.8 




Frustration 


100.0 


100.0 


98.0 


98.7 


99.2 


4 


Independent 


0.0 


0.0 


0.9 


0.0 


0.3 




Instructional 


0.0 


6.1 


2.6 


0.0 


2.4 




Frustration 


100.0 


93.9 


97.4 


100.0 


97.6 


5 


Independent 




1.0 


4.5 


0.0 


1.9 




Instructional 




7.3 


29.5 


19.8 


20.3 




Frustration 




92.7 


70.5 


80.2 


79.7 


6 


Independent 






8.0 


0.0 


3.8 




Instructional 






33.3 


28.2 


30.6 




Frustration 






66.7 


71.8 


69.4 


7 


Independent 








1.5 


1.5 




Instructional 








33.8 


33.8 




Frustration 








66.2 


66.2 





Explicit Reading Comp. --Percent Correct ^ Explicit Reading Comp.--Percent Correct 
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lire 40. Explicit Reading Comprehension Performance (% Correct), by Grade RM began — Expository (Silent) Reading 
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Figure 41. Explicit Reading Comprehension Performance (% Correct), Combined All Students by Grade Level — Expository (Silent) Reading 
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Table 18. 

Explicit Reading Comprehension Performance by Grade Level — Expository (Silent) Reading 



Beginning Grade of RM 


Grade 




K 


1 


2 


3 


Combined 


K 


Mean 


15.54 








13.71 




S.D. 


18.55 








18.55 




N 


428 








350 


1 


Mean 


15.91 


12.23 






13.70 




S.D. 


22.05 


19.99 






21.43 




N 


264 


94 






302 


2 


Mean 


26.28 


23.13 


26.35 




23.95 




S.D. 


26.78 


23.39 


24.81 




25.22 




N 


176 


87 


102 




311 


3 


Mean 


23.99 


25.30 


21.20 


23.27 


22.79 




S.D. 


22.79 


23.81 


19.23 


21.40 


21.70 




N 


99 


82 


92 


65 


289 


4 


Mean 


36.97 


46.61 


25.39 


29.75 


32.86 




S.D. 


26.93 


25.81 


23.87 


18.59 


22.91 




N 


47 


70 


96 


79 


245 


5 


Mean 




66.10 


72.60 


63.41 


67.74 




S.D. 




26.25 


24.80 


24.98 


24.90 




N 




59 


73 


82 


155 


6 


Mean 






65.67 


68.40 


68.40 




S.D. 






25.76 


26.78 


26.78 




N 






75 


72 


72 


7 


Mean 








44.85 


44.85 




S.D. 








23.88 


23.88 




N 








68 


68 
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Figure 42. Implicit Reading Comprehension Performance (% Correct), by Grade RM began — Expository (Silent) Reading 




Figure 43. Implicit Reading Comprehension Performance (% Correct), Combined All Students by Grade Level — Expository (Silent) Reading 
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Table 19. 

Implicit Reading Comprehension Performance (% Correct,) by Grade Level — Expository (Silent) Reading 



Beginning Grade of RM 


Grade 




K 


1 


2 


3 


Combined 


K 


Mean 


14.50 








14.50 




S.D. 


24.13 








24.13 




N 


350 








350 


1 


Mean 


18.75 


31.90 






22.84 




S.D. 


28.29 


33.28 






29.93 




N 


208 


94 






302 


2 


Mean 


26.43 


12.07 


19.85 




20.25 




S.D. 


39.60 


15.30 


20.46 




28.60 




N 


122 


87 


102 




311 


3 


Mean 


41.50 


49.82 


23.78 


38.08 


37.42 




S.D. 


24.15 


27.71 


18.90 


21.47 


23.16 




N 


50 


82 


92 


65 


289 


4 


Mean 


30.85 


29.46 


21.35 


15.66 


21.83 




S.D. 


25.58 


25.54 


19.86 


13.64 


20.02 




N 


47 


70 


96 


79 


245 


5 


Mean 




42.80 


45.03 


44.51 


44.75 




S.D. 




25.17 


25.41 


19.45 


22.45 




N 




59 


73 


82 


155 


6 


Mean 






42.33 


38.02 


38.02 




S.D. 






24.69 


22.93 


22.93 




N 






75 


72 


72 


7 


Mean 








50.00 


50.00 




S.D. 








23.78 


23.78 




N 








68 


68 
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Table 20. 

Total Reading Comprehension Performance (Explicit and Implicit) by Grade Level, Percent Correct 



Beginning Grade of RM 


Grade 




K 


1 


2 


3 


Combined 


K 


Mean 


15.14 








15.14 




S.D. 


16.87 








16.87 




N 


452 








452 


1 


Mean 


17.45 


18.79 






17.78 




S.D. 


18.55 


19.97 






18.91 




N 


288 


94 






382 


2 


Mean 


26.02 


17.60 


23.10 




23.34 




S.D. 


16.35 


15.32 


18.63 




16.77 




N 


196 


87 


102 




385 


3 


Mean 


32.22 


37.50 


22.49 


30.67 


30.61 




S.D. 


15.55 


21.09 


16.18 


17.32 


17.48 




N 


110 


82 


92 


65 


349 


4 


Mean 


33.91 


38.04 


23.37 


22.71 


28.40 




S.D. 


16.11 


20.41 


19.16 


12.43 


13.76 




N 


47 


70 


96 


79 


292 


5 


Mean 




34.85 


58.82 


53.96 


50.35 




S.D. 




16.86 


21.09 


17.33 


18.58 




N 




59 


73 


82 


214 


6 


Mean 






54.00 


53.21 


53.61 




S.D. 






16.75 


20.82 


18.85 




N 






75 


72 


147 


7 


Mean 








47.43 


47.43 




S.D. 








15.65 


15.65 




N 








68 


68 





